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ABSTRACT 

Two  important  enabling  but  evolving  technologies 
supporting  future  DoD  netH’ork-centric  systems  at 
the  tactical  edge  are,  mobile  ad  hoc  net\\>orking 
(MANET)  and  Multi-Agent  Systems  (MAS). 
Despite  their  value  in  enabling  more  autonomous 
netw’ork  system  operation  scenarios,  open 
research  and  engineering  questions  remain 
regarding  robust  interoperation,  standardization, 
and  design  of  these  two  technologies.  We  describe 
recent  research  and  development  that  is  helping  to 
better  understand  crosslayer  design  issues  within 
both  MAS  and  MANET.  We  describe  the  problem 
area  and  the  open  software  components  developed 
to  support  our  research.  We  summarize  recent 
modeling  and  simulation  advances  in  a  mixed  MAS 
and  MANET  scenario  environment.  MANET 
multicast  approaches  for  interagent 
communications  are  discussed  and  described. 
Some  early  analysis  of  MAS  performance  is 
presented  using  a  variety  of  interagent  MANET 
communication  models.  The  behavior  of  MAS 
autonomous  cooperative  teamwork  and  role 
allocation  within  disruptive  and  dynamic  MANET 
environments  is  examined.  We  conclude  by 
outlining  open  issues  and  areas  of  further  work. 

BACKGROUND  AND  MOTIVATION 

There  is  planned  deployment  of  Mobile  Ad  hoc 
Networking  (MANET)  type  network  routing 
technology  at  the  battle  space  forward  edge  or  the 
"the  first  tactical  mile".  There  is  also  early 
deployment  of  agent-based  systems  occurring  now 
with  more  extensive  future  deployment 
envisioned.  At  present,  there  remains  a  limited 
understanding  of  appropriate  architectural  design 
tradeoffs  in  adapting  upper  layer  protocols  and 
applications  in  these  environments.  Also  previous 
design  work  done  with  agent  based  networking 
and  software  has  often  assumed  benign  network 


behavior  and  highly  stable  infrastructures  not 
MANET  environments.  Figure  1  provides  a  high 
level  depiction  of  some  of  the  common 
characteristics  differentiating  mainstream  Internet, 


Figure  1 :  Network  Problem  Area  Focus 

high  performance  network,  and  tactical  edge  type 
environments.  The  characteristics  depicted  in 
Figure  1  emphasize  why  focused  research  should 
continue  to  be  performed  on  tactical  edge  solutions 
including  a  multilayered  examination  of 
perfonnance  issues  and  tradeoffs. 

PROBLEM  FOCUS 

Our  technical  focus  is  to  improve  design  and 
performance  of  both  MANET  and  MAS 
technology  within  dynamic  network  centric 
problem  scenarios. 

Some  of  the  technical  goals  include  the  following: 

-  Investigate  MAS  design  robustness  in 
stressed,  mobile  network  environments 

-  Develop  network  protocol  enhancements 
and  improve  modeling  to  study  behavior 
where  needed 

-  Improve  design  tradeoff  understanding  by 
studying  a  combined  solution  (not  well 
examined  to  date) 


1 


Report  Documentation  Page 

Form  Approved 

0MB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
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-  Develop  working  models  and  software  that 
can  more  easily  transition 
The  core  technical  challenge  of  our  work  involves 
tackling  cross-disciplinary  issues  of  dynamic 
network  protocol  and  multi-agent  system  design. 
Separately,  MAS  and  MANET  encompass  two 
challenging  research  and  engineering  areas.  Wlien 
considered  as  a  combined  technical  solution,  the 
challenges  increase  due  to  complex  crosslayer 
design  issues  and  behaviors  [CDCP05].  We 
summarize  these  challenges  as  follows: 


MANET  Project-Related  Challenges: 

•  Rich  taxonomy  of  design  choices  exist 

•  Present  lack  of  modeling  and  support  for 
interagent  communication  components 

•  Crosslayer  performance  design  issues 

•  Possible  agent  and  middleware  self¬ 
organization  interactions  given  MANET 
behaviors  and  dynamics 

•  Protocol  scalability  and  robustness 
MAS  Project-Related  Challenges: 

•  Rich  taxonomy  of  design  choices  exist 

•  Lack  of  modeling  and  analysis  in  stressed 
network  environments 

•  Adaptation  of  well-known  strategies  in 
disruptive  MANET  environments 

•  Designs  supporting  efficient  inter-agent 
communications 

PREVIOUS  WORK 

Ongoing  research  and  development  is  being  done 
in  both  MANET  [B04][CM99][P01]  and  MAS 
[W02]  technology  areas  by  numerous  parties.  We 
are  leveraging  existing  and  previous  work 
accomplishments  to  execute  our  research  goals. 
Our  previous  accomplishments  in  this  area  were 
documented  in  [MAMC05].  Here  we  update  our 
research  approach  and  results  and  provide  more 
detailed  evaluation  of  cooperative  multi-agent 
scenarios  using  specific  MANET  protocol 
technology. 

MODELING  APPROACH  AND  ISSUES 

Figure  2  illustrates  the  modeling  system 
requirements  and  related  issues  that  were  required 
in  developing  a  framework  for  planned  research 
activities.  A  fundamental  need  was  to  develop 


distributed  MANET  modeling  capabilities  in  both 
simulation  and  emulation  and  this  required  the 
following  components: 

•  Means  of  producing  and  controlling 
network  dynamics  and  node  mobility 

•  MANET  protocol  prototypes  and  the 
potential  for  supporting  middleware 
system  components 

•  Multi-agent  software  prototypes  with 
flexible  network  interface  components 

•  Environmental  model  interface  to  the 
distributed  agents 


Figure  2:  System  Modeling  Requirements 


RECENT  MODELING  PROGRESS 

Figure  3  depicts  a  system  diagram  of  the  modeling 
progress  accomplished  in  both  simulation  and 
emulation  environments.  The  Agent  Toolkit  has 
been  recently  developed  to  allow  a  more  cross 
platform  and  extensible  approach  to  be  pursued  in 
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Figure  3:  Key  Component  Modeling  Diagram 

developing  and  enhancing  java  software  agents 
placed  within  our  test  environments  for 
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experimentation.  AgentJ  [ADMT06]  and  the 
Protolib  software  components  shown  in  Figure  3 
allowed  for  standard  java  code  to  be  ported  and 
implemented  within  the  NS-2  simulation 
environment.  This  has  enabled  more  flexibility  and 
less  abstraction  in  simulating  application  layer 
components.  This  will  also  likely  ease  the 
inclusion  of  additional  application  agents  in  future 
efforts.  As  also  shown  in  Figure  3,  a  distributed 
environmental  simulation  model  and  agent 
interface  was  added  to  the  NS-2  environment  to 
provide  external  dynamic  agent  stimulus  and 
environmental  modeling  input.  This 

environmental  modeling  channel  and  approach  can 
be  used  to  represent  external  sensor  systems  and 
other  dynamic  environmental  phenomena  [D04]. 
Recent  modeling  advancements  go  beyond 
previous  reported  results  allowing  us  to  scale  our 
experiments  and  include  additional  capabilities 
such  as  MANET  multicast  routing  and  more 
complex  stimulus. 

RECENT  MAS  AND  MANET  STUDIES 
We  have  constructed  a  number  of  dynamic 
problem  solving  scenarios  that  exercise  both  multi¬ 
agent  system  interactions  and  the  underlying 
MANET  communications  on  the  move.  An 
example  of  one  of  our  primaiy^  agent  team-based 
problem  solving  scenarios  is  a  prey/predator 
pursuit  problem.  The  prey/predator  problem  is  a 
canonical  example  of  agent  teamwork  in  the 
literature.  We  adapted  this  problem  to  further 
study  interagent  behavior  and  teamwork 
performance  and  have  included  MANET  routing 
and  communications  support. 

A  challenge  in  carrying  out  planned  experiments  is 
the  actual  data  collection  and  measurement  of 
system  effectiveness.  While  we  routinely  collect 
and  are  interested  in  detailed  protocol  and  network 
statistics,  we  are  now  also  interested  in  measures 
of  effectiveness  for  MAS.  Since  we  are  studying 
dynamic  role  allocation  and  agent  teamwork  we 
developed  initial  approaches  to  measure 
coordination  quality.  Given  a  scenario,  one  metric 
we  measure  is  the  time  or  number  of  steps  it  takes 
to  complete  a  task  or  set  of  tasks.  This  provides  a 
rough  effectiveness  measure  of  teamwork 
performance  given  a  particular  system  design.  We 
also  gather  and  examine  statistics  in  a  coordination 
metric  that  measures  interactions  between  agents 
performing  distributed  role  allocation  (e.g.. 


number  of  messages  received  and  role  collision 
events). 

In  previous  work  [ACM05],  we  reported  on  the 
initial  examination  of  prey/predator  agent  systems 
within  simulated  networks  in  the  Repast  agent- 
based  modeling  tool  [NCV06].  In  recent 
experiments,  we  have  performed  scaled  studies  (> 
60  agents)  in  NS-2  simulation  to  examine  large 
agent/team  performance.  This  is  done  while 
maintaining  highly  detailed  protocol  and 
application  modeling  capturing  complex 
behavioral  and  performance  interactions  across 
layers.  Figure  4  shows  a  snapshot  visualization  of 
a  30  predator  agent  and  30  prey  agent  simulation 
scenario  recently  executed.  This  simulation 
scenario  involves  60  fully  functional  java  agents, 
of  which  30  predator  agents  operate  with  a  highly 
detailed  MANET  and  IP  network  protocol  stack 
model.  This  test  also  includes  the  operation  at 
each  node  of  a  MANET  unicast  and  multicast 
forwarding  protocol  to  enable  a  self-organizing 
network  even  during  periods  of  topology 
fragmentation. 

Our  predator/prey  teamwork  researeh  approach 
also  involves  examining  differing  interagent 
communication  requirements  and  eapabilities  that 
match  various  role  alloeation  algorithms  being 
examined.  As  reported  in  [ACM05]  we  used  an 
analytical  model  to  examine  a  range  of  role 
allocation  strategies  that  require  different  types 
and  levels  of  interagent  network  messaging.  An 
early  set  of  results  demonstrated  that  the 
Distributed  Constraint  Optimization  (DCO) 
approach  based  upon  the  Hungarian  method 
[K55][PS98]  outperfomied  other  role  allocation 
methods  in  simulation.  We  are  examining  this 
same  method  in  a  more  detailed  scaled  network 
scenario  with  network  congestion  and  dynamic 
topology  disruption  as  additional  parameters. 

The  basic  prey/predator  scenarios  being  used  can 
be  described  as  follows.  The  scenario  is  initialized 
with  a  number  of  predators  and  preys  in  random 
geographic  locations.  Predators  sense  preys  using 
an  abstracted  sensor  model  and  can  communicate 
with  other  predators  using  MANET  networking 
supported  within  the  NS-2  framework.  The 
limited  sensing  and  wireless  communication  range 
of  each  predator  provides  both  a  limited 
environment  view  and  a  limited  directly  connected 
communication  neighborhood.  The  scenario  tasks 
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require  at  minimum  N  predators  to  surround  and 
therefore  capture  each  prey.  Although  the  multiple 
prey  team  and  surrounding  requirements  establish 
a  simple  role  allocation  problem,  the  scenario  is 
highly  dynamic  and  uses  detailed  application  and 
networking  models  to  cany'  out  an  experiment. 

Due  to  limited  communication  capabilities  and 
ranges,  interagent  networking  is  needed  to  provide 
multihop  connectivity.  The  predator  network  may 
also  undergo  unpredictable  fragmentation  and 
merging  due  to  movement  required  to  accomplish 
the  task.  To  support  research,  most  protocol  and 
environment  conditions  are  parameterized  and 
some  examples  include  the  following: 

•  Sensor  ranges  and  qualities  for  each  prey 
and  predator  can  be  programmed.  Special 
nodes  may  be  added  to  have 
heterogeneous  sensing  capability. 

•  Differing  speeds  and  capabilities  can  be 
established  for  each  agent.  Agents  may  be 
programmed  to  remain  fixed  if  desired. 

•  Predator  agents  have  limited  wireless 
communication  range  that  is 
programmable.  Other  environment  effects 
and  propagation  models  can  be  added  to 
the  scenario. 

•  A  variety  of  MANET  networking 
protocols  and  transport  methods  can  be 
used  by  the  agents  during  experimentation. 
Fragmentation  and  coalescing  of 
connected  networks  occurs  during 
scenarios  and  is  supported. 

•  Predators  must  form  teams  of  minsize  =  d, 
pursue  and  surround  a  prey  to  accomplish 
a  successful  capture.  These  parameters 
can  be  adjusted,  but  the  default  team  size 
is  four. 

•  High  level  tasks  (e.g.,  capture)  can  be 
defined  and  the  number  of  steps  can  be 
measured  along  with  other  metrics  and 
network  protocol  statistics. 

•  Different  network  protocols,  role 
allocation  strategies,  and  external  network 
disniption  can  be  added  to  create  different 
scenarios. 

Figure  4,  shows  a  screenshot  from  an  example 
multi-agent  simulation.  The  links  drawn 
demonstrate  the  dynamic  topology  that  is  possible 


between  the  agent  nodes.  Tliese  dynamic  links  are 
shown  between  cooperative  predator  agents 
dynamically  forming  ad  hoc  networks  and 
perfomiing  cooperative  task  allocation  to  capture 
prey  (red  denotes  captured,  green  denotes 
uncaptured).  Scout  nodes  are  stationary,  but 
contribute  to  predator  task  allocation  by  providing 
long-range  prey  detection. 


time=20:00 


Figure  4:  Multi-Agent  Predator  Teamwork 
Simulation 

MANET  MULTICAST  FOR  INTRA-AGENT 
COMMUNICATIONS 

In  order  to  provide  enhanced  network  routing  and 
network  messaging  mechanism  for  agent 
teamwork  communications  within  MANETs,  we 
are  applying  related  research  and  prototype 
development  of  Simplified  Multicast  Forwarding 
(SMF).  SMF  research  and  emerging  IETF 
specifications  are  discussed  in  [M06]  and 
[MDC04].  SMF  provides  a  simple  multicast 
forwarding  capability  for  application  data  flows 
witliin  a  MANET  routing  area.  SMF  also  provides 
a  foiw'arding  process  compatible  with  different 
neighborhood  discovery  protocols  and  optimized 
relay  set  selection  algorithms.  In  MAS  teamwork 
role  allocation  strategies,  group  communication 
and  maintenance  of  either  environmental  or  agent 
decision  state  is  often  required  to  be  shared.  SMF 
provides  an  efficient  MANET  networking 
mechanism  to  quickly  forward  this  state  in  a 
dynamic  environment.  In  one  class  of  role 
allocation  strategy,  the  distributed  stochastic 
algorithm  (DSA)  [FMOl],  role  changes  are  made 
stochastically  by  the  agents  to  reduce  conflicts  and 
communicated  until  a  consistent  solution  is 
obtained.  In  a  second  class,  as  in  the  distributed 
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coordination  optimization  (DCO),  the  state 
information,  or  beliefs,  is  communicated  between 
agents  to  infer  their  respective  role  using  a 
constraint  optimization  method.  We  are  examining 
all  these  classes  within  our  research.  In  all  these 
cases,  while  different  information  is  being  shared 
amongst  agents  there  is  a  common  requirement  for 
communications  of  either  state  or  intent.  The 
application  of  SMF  provides  an  effieient 
networking  means  of  routing  this  group-based 
interagent  communication  within  a  highly  dynamic 
network  environment.  While  SMF  is  efficient  if 
many  agents  require  the  same  message,  future 
work  will  study  the  system  design  tradeoffs  as 
communication  overhead  and  congestion 
conditions  increase  within  our  network  scenarios. 

CASE  STUDY:  WIRELESS  NRL 
EMULATOR  EXPERIMENTS 

Factorial  experiments  were  conducted  in  the 
wireless  NRL  emulator  [CMW03]  to  study  the 
impact  of  various  parameters  on  the  coordination 
performance  of  multi-agent  systems  in  MANET. 
Using  the  prey/predator  scenario  described  above 
and  a  fixed  network  density  (10  nodes),  the 
performance  was  measured  in  terms  coordination 
[ACM05]  and  robustness  (coefficient  of  variation 
w.r.t  the  median  number  of  steps  required  to 
accomplish  the  goal).  This  robustness  metric 
encapsulates  the  trade-off  between  performance 
and  reliability.  The  communication  range  was 
allowed  to  vary  in  the  100-300  meters  range  to 
better  reflect  heterogeneous  environments.  Other 
parameters  such  as  the  speed  of  the  agents  were 
also  allowed  to  vary.  Figures  5  and  6  show 
comparative  experimental  results  in  different 
congestion  environments  for  different  role 
allocation  algorithms.  These  results  show  how 
performance  rapidly  degrades  as  congestion 
increases. 

The  algorithms  analyzed  in  these  figures  invoke 
different  random  processes  for  solving  the 
constraint  optimization  problems  in  our 
prey /predator  scenarios.  DCO  70%  will 
communicate  status  messages  used  to  resolve 
conflicts  only  70%  of  the  time.  In  the  case  of 
DSA,  a  role  change  to  resolve  conflicts  will  occur 
only  70%  of  the  time,  and  if  a  role  change  occurs, 
it  will  be  communicated  to  the  other  agents. 


Coordination  Performance  in  Wireiess  Emuiator 


Figure  5:  Comparative  coordination 
performance  at  different  congestion  levels 


Robustness  Performance  in  Wireiess  Emuiator 


Figure  6:  Comparative  robustness  at 
different  congestion  levels 


CASE  STUDY:  INTERAGENT  AD  HOC 
NETWORKING  VALUE 

Using  and  applying  the  detailed  modeling 
components,  agent  role  alloeation  designs,  and 
MANET  protoeols  previously  mentioned,  we 
present  Figure  7  to  illustrate  how  communication 
networking  can  improve  agent  task  completion.  In 
figure  7  along  the  x  axis  we  start  out  with  a 
slightly  fragmented  physieal  network  topology. 
Under  these  eonditions,  we  measure  the  time  or 
steps  it  takes  for  the  agent  group  to  complete  the 
prey  eapturing  task.  In  subsequent  experiments, 
we  increase  the  transmit  power  parameter  or 
communieation  range  achievable  by  predator 
agents,  thus  inereasing  the  probability  of  forming  a 
physieally  eonneeted  network  topology.  From  the 
eurve,  we  illustrate  the  variance  of  particular 
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measurements  and  the  fact  that  as  networking 
probability  increases  the  agent  coordination 
generally  improves.  As  network  density  increases 
we  have  also  shown  from  related  SMF  research 
that  we  can  control  the  amount  of  network  traffic 
overhead  growth  for  multicast  data  flows  to 
reasonable  levels. 
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Figure  7:  Task  Completion  vs.  Network 
Likelihood 

While  this  is  only  one  basic  example,  it 
demonstrates  tlrat  there  are  performance  gains 
from  networking  agents.  It  also  indicates  that 
these  gains  can  be  measured  in  detailed  modeling 
environments.  Environmental  conditions,  sensor 
models,  agent  coordination  strategies,  and 
MANET  protocol  parameters  can  all  be  adjusted 
accordingly  to  examine  different  design  and 
scenario  tradeoffs.  Further  detailed  scenarios 
specific  studies  are  largely  tire  scope  of  fiiture 
applied  work  that  is  planned. 

FUTURE  WORK 

Part  of  the  challenge  of  studying  MAS  systems  is 
developing  approaches  and  actually  measuring  the 
effectiveness  of  performance.  This  is  even  more 
challenging  within  highly  complex  environments 
with  competing  goals.  Using  the  outlined 
simulation  and  emulation  test  environments 
discussed  here,  our  future  work  will  further 
explore  a  set  of  use  case  scenarios  to  exercise 
MAS  and  MANET  protocol  performance. 

CONCLUSION 

We  have  briefly  described  recent  research  and 
development  that  is  examining  crosslayer  issues 
which  affect  performance  of  MAS  techniques 
operating  within  MANET  environments.  We 
summarized  our  recent  modeling  and  simulation 
advances  in  achieving  a  mixed  MAS  and  MANET 


scenario  enviromnent  that  can  be  used  for  further 
complex  analysis.  MANET  multicast  approaches 
for  interagent  communications  were  also  briefly 
discussed  and  results  from  using  this  technique 
showed  significant  agent  system  improvements  as 
networks  became  less  fragmented  and  were  able  to 
take  advantage  of  multicast.  We  presented  furtlier 
emulation  and  simulation  analysis  of  MAS 
autonomous  cooperative  teamwork  and  role 
allocation  within  dismptive  and  dynamic  MANET 
environments.  While  interagent  communications 
require  additional  network  overheard  and  are 
prone  to  dismption  within  a  MANET  scenario, 
early  results  from  our  work  indicate  that 
appropriate  crosslayer  design  approaches  can 
improve  MAS  perfonnance.  Further  work  has 
been  outlined  to  continue  more  complex  scenarios 
and  the  range  of  design  tradeoffs. 
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